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DETAILED ACTION 



Response to Amendment 



1. 



This Office Action is responsive to the amendment filed May 19, 2008. 



2. 



Claims 1-25 are pending. 



Response to Arguments 



3. Applicant's arguments filed May 19, 2008 have been fully considered but they are not 
persuasive. 

4. Applicant argues that Tyree does not teach "requiring a computer user to locate at least 
one feature of said one or more deformed body parts in the image". 

5. Applicant argues that Mori does not teach "requiring a computer user to locate at least 
one feature of said one or more deformed body parts in the image". 

In response to applicant's arguments against the references individually, one cannot show 
nonobviousness by attacking references individually where the rejections are based on 
combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re 
Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

6. The claim recites "generating a human interactive proof employing an image of one or 
more deformed body parts wherein certain features thereof are at known locations in said image. 
However, the claim does not define the phrase "certain features" and the specification does not 
provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would 
not be reasonably apprised of the scope of the invention. What are these certain features? 
Further, the claim recites "known locations", but no locations were previously identified? What 
are these locations? 
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7. Applicant argued that "there is nothing to teaching, motivation or suggestion to combine 
the teachings of Tyree and Mori". The Examiner notes that KSR forecloses the argument that a 
specific teaching, suggestion, or motivation is required to support a finding of obviousness. See 
KSR, 127 S. Ct. at 1741, 82 USPQ2d at 1396. 

8. Applicant argues that Mori does not suggest locating key points in a distorted body part. 
However, the Examiner respectfully disagrees. Figure 5 of Tyree is an illustration of an 

intelligence test that can be presented to a user. In Tyree, a distorted image is generated and is 
presented with a distorted image (see Fig. 5, 502 & paragraph [103] - the graphical image can be 
generated so as not to be easily recognized using image recognition technology). A challenge 
prompting and querying the user to input is provided, which is a requirement. The inputted 
information is compared and it is determined whether or not the user is a human or computer 
program. Mori discloses an algorithm for locating key points within an image of a body (see 
page 1). It would be obvious at the time the invention was made to substitute the body image of 
Mori with a distorted image because the simple substitution of one known element for another 
producing a predictable result renders the claim obvious. 

9. Applicant argues that neither Tyree nor Mortlock teaches the limitations of Claim 23. 

In response to applicant's arguments against the references individually, one cannot show 
nonobviousness by attacking references individually where the rejections are based on 
combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re 
Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

10. Applicant argued that there is nothing to teaching, motivation or suggestion to combine 
the teachings of Tyree and Mortlock. The Examiner notes that KSR forecloses the argument that 
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a specific teaching, suggestion, or motivation is required to support a finding of obviousness. 
See KSR, 127 S. Ct. at 1741, 82 USPQ2d at 1396. 

Claim Rejections - 35 USC §103 

1 1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

12. Claims 1-4, 7-11, 13, 17-19, 21 and 22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Tyree US PGPUB 2002/0120853 (hereinafter referred to as Tyree) in view of 
Greg Mori et al. "Estimating Human Body Configurations using Shape Context Matching" from 
the 2001 IEEE Computing Society's Conference on Computer Vision and Pattern Recognition 
(hereinafter referred to as Mori). 

Claim 1 of the application recites: 

A computer-implemented process for determining whether a computer user is a human or 
a computer program, comprising the process actions of: 
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generating a human interactive proof ( [0034], [0039], [0072], [0103]) employing an 
image ([0029], [0102], [0103]) of one or more body parts wherein certain features thereof 
are at known locations in said image; 

requiring a computer user to locate ([0029]) at least one feature of said one or more body 
parts in the image; 

comparing the computer user's locations of said at least one feature of said one or more 
body parts to their actual location in the image; and 

determining whether the computer user is a human or a computer program ([0042], 
[0072], and [0074]). 

Tyree discloses the exemplary embodiment of an intelligence test which offers an image 
and prompts the user to answer a question about said image ([0102]), which is then compared 
with the stored answer ([0103]). It does not however discuss an image which contains body 
parts, and location of such parts. The security of any human interactive proof is contingent upon 
the ability of the algorithm to correctly identify elements within the produced tests for 
comparison with user input. The Mori paper discloses an algorithm for locating key points within 
an undistorted image of a body (paragraph 1, pg 1). The basic image then would be 
understandable to the program prior to its distortion. After transformations (which are known to 
the program) are applied, the resultant image would not be identifiable by body/facial 
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recognition software of the time. As for feature where the body parts are deformed, Tyree 
generating and presenting distorted graphical images (see paragraph [0103]). It would be obvious 
at the time the invention was made to substitute the body image of Mori with a distorted image 
because the simple substitution of one known element for another producing a predictable result 
renders the claim obvious. Also, it would have been obvious to one of ordinary skill in the art at 
the time of invention to combine the two in order to increase the level of security offered by the 
"test", thus improving the marketability of the software product as a whole. 

Referring to claim 2, Mori discloses wherein said deformed body parts is a human face 
see section 5, pg 5 and claim 1 above). 

Claim 3 of the application recites: 

The computer-implemented process of Claim 1 wherein said one or more deformed 
body parts is an entire human body. 

The additional limitation of a human body is taught by Mori (section 1 , paragraph 1 , pg 1 
and claim 1 above). 

Claim 4 of the application recites: 

The computer-implemented process of Claim 1 wherein said one or more 
deformed body parts is an animal. 
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The additional limitation of an animal is disclosed by Mori (section 1, paragraph 1, pg 1; 
claim 1 above). 

Claim 7 of the application recites: 

The computer-implemented process of Claim 1 wherein the process 
action for determining whether the computer user is a human or a computer program 
comprises using a comparison of the computer user's locations of said at least one feature 
of said one or more body parts to the location of said features in the image. 

Tyree teaches a system which includes a comparator for comparing the expected answer 
of a given "test" with the answer provided by the user (claim 18). Utilizing the images of body 
parts disclosed earlier by Mori, the challenge put forth by the validating program ([0102]) could 
reasonably consist of identifying and locating parts of the image. It would have been obvious to 
one of ordinary skill in the art at the time of the invention to combine the system of Tyree with 
the recognition system of Mori, to improve the marketability of the software product by 
providing a slightly different form of test challenge. 

Claim 8 of the application recites: 

The computer-implemented process of Claim 1 wherein the computer- 
user-identified feature locations are specified by the user, using a computer pointing 
device. 
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The limitation of identifying locations in an image using an input device is met by Tyree 
who discloses querying a user for user input regarding the challenge, within an input field, which 
can reasonably be construed as the clickable region of an image map. Furthermore Tyree 
describes computer having input devices which include a mouse or other pointing device 
([0096]), thus meeting the additional limitation of a computer pointing device. Mori's algorithm 
discloses automated feature location (section 1, paragraph 1, pg 1). It would have been obvious 
to one of ordinary skill in the art to combine the two to improve efficiency of the human 
interactive proof by allowing a user to click on features rather than typing them out. 

Claim 9 of the application recites; 

The computer-implemented process of Claim 8 wherein the computer 
Pointing device is one of: 
a mouse; and 
a digital pen. 

The limitation of a pointing device is taught by Tyree 's disclosure of computer input 
devices ([0096]). 

Claim 10 of the application recites: 

A system for creating a Human Interactive Proof using an image of a face, 
the system comprising: 

a general purpose computing device ([0092] - [0095]); and 
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a computer program comprising program modules executable by the computing 
device, wherein the computing device is directed by the program modules of the 
computer program ([0098] - [0099]) to, 

generate a human interactive proof employing an image ([0029], [0034], [0072], and 
[0103]) of a deformed human face wherein certain features are at known locations in said 
image; 

require a computer user to locate ([0029]) certain features of said deformed face in the 
image; 

compare the computer user's locations of said features of said deformed face to their 
actual location in the image; and 

determine whether the computer user is a human or a bot ([0042], [0072], and [0074]). 

Tyree discloses an intelligence test which offers an image to a potential user and prompts 
the user to answer a question about said image ([0102]), which is then compared with the stored 
answer ([0103]). It does not disclose an image which contains a human face, and location of 
facial features. Mori paper discloses an algorithm for locating key points within an undistorted 
image of a body (paragraph 1, pg 1), which can be applied to facial recognition (item 1, section 
5, pg 5). The basic image then would be understandable to the program prior to its distortion. 
After transformations (which are known to the program) are applied, the resultant image would 
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not be identifiable by body/facial recognition software of the time. As for feature where the 
body parts are deformed, Tyree generating and presenting distorted graphical images (see 
paragraph [0103]). It would be obvious at the time the invention was made to substitute the body 
image of Mori with a distorted image because the simple substitution of one known element for 
another producing a predictable result renders the claim obvious. It would have been obvious to 
one of ordinary skill in the art at the time of invention to combine the two in order to increase the 
level of security offered by the "test", thus improving the marketability of the software product 
as a whole. As for 

Claim 1 1 of the application recites: 

The system of Claim 10 wherein the image is automatically synthesized. 

Tyree does not explicitly disclose automated image synthesis it does discuss a random 
test generator that determines an expected answer ([0039]). The Mori paper teaches an algorithm 
for automatically determining features on an image of a human body. It is necessary for the test 
generator of Tyree to have the capability of determining the answer to the test, in this case the 
location of body parts within the image. It would have been obvious to one of ordinary skill in 
the art at the time of the invention to combine the test generator of Tyree with the automated 
process of Mori in order to improve efficiency. 



Claim 13 of the application recites: 
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The system of Claim 10 wherein the module to determine whether a 

computer user is a human or a bot determines that the computer user is a human if the 

computer user's locations of said features are within a given distance from their actual 

location. 

The only additional limitation claim 13 adds to claim 10 is that of the locations of 
features, which are input by the user, as being within a given distance from their actual location. 
With regards to a comparison of feature location, there are three possible outcomes for a user's 
estimated location: it is either within a given distance, outside a given distance, or at the origin/at 
the actual location. Since a comparison of the estimated features is being made to their actual 
location already it would be obvious to one of ordinary skill in the art at the time of the invention 
to conduct that comparison based on the estimate being within a given distance. 

Claim 17 of the application recites: 

The system of Claim 10 wherein the determination of whether the user is 
a human or a computer program is made without human intervention. 

The limitation of lack of human intervention is taught by Tyree as a comparator 
component of the system ([0039]). 



Claim 18 of the application recites: 
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The system of Claim 10 wherein the user points to the feature points with 
a computer input device. 



Additionally Claim 19 recites: 

The system of Claim 18 wherein the computer input device is a mouse. 

The Tyree publication discloses the limitation of identifying locations in an image using 
an input device by querying a user for input regarding the challenge, within an input field, which 
can reasonably be construed as the clickable region of an image map. 

Claim 21 of the application recites: 

The system of Claim 10 wherein the image size is 512 x 512 pixels. 

The examiner takes official notice that it would have been obvious to one of ordinary 
skill in the art of image manipulation at the time of the invention to generate, crop, resize, or 
otherwise manipulate the image of claim 10 to the size of 512 x 512 pixels. 

Claim 22 of the application recites: 

The system of Claim 10 wherein the image in the human interactive proof is test image IF 
with ground truth of face locations and facial feature locations. 
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The scope of this claim is covered under claim 10's disclosure of generating an HIP 
employing the image of a human face and having known locations of features within the image. 
These feature locations are used by the system for comparison ([0034]), to determine 
authentication and thus it is inherent that ground truth should be represented by these locations. 
This claim encompasses no additional matter and is therefore rejected under the same premise as 
claim 10. 

13. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Tyree in view of 
Mori and further in view of Luis von Ahn, ct al "CAPTCHA: Using Hard AI Problems For 
Security" (hereinafter referred to as von Ahn), presented at the EuroCrypt 2003 conference in 
Warsaw Poland. 

Claim 5 of the application recites: 

The computer-implemented process of Claim 1 wherein said 

determination of whether a computer user is a human or a computer program is used for 
one of: 

assigning an email account; 

validating an input in a poll; 

using a search engine; 

using a chat room; and 

accessing data on a website. ([0036]) 
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Tyree teaches the concept of using a human interactive proof to control access to network 
resources and storage ([0036]). The von Ahn paper goes further to describe the use of HIP for 
Online polls (pg 1, section 1, paragraph 2), Email account services (section 1, paragraph 1, pg 2), 
and Search Engines (section 1, paragraph 2, pg 2). All of these are examples of network 
resources and storage facilities. It would have been obvious to one of ordinary skill in the art at 
the time of the invention to utilize the HIP for controlling access to many types of internet 
accessible resources to expand the usefulness of the proof generating software and thus improve 
its overall marketability 

14. Claims 12, 16, 20 and 23-25 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Tyree in view of Mortlock et al, US Patent 6,549,200 (hereinafter referred to as Mortlock). 

Claim 12 of the application recites: 

The system of Claim 10 wherein the image is a distorted face embedded 
in a cluttered background 

The applicant discloses that it is known in the art to put an image of a distorted word on a 
cluttered background image ([0006]). The applicant does not disclose the image of a face on a 
cluttered background. The Mortlock patent discloses the ability to create an image of a human 
head/face. It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combine the applicant's disclosed image, with that of Mortlock s facial image in 
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order to expand the variety of images that can be used in an HIP system. Increasing the variety of 
usable images will increase security offered by the system and therefore its marketability. 

Claim 16 of the application recites: 

The system of Claim 10 wherein the image is generated to include at least 
one of: 

non-frontal faces; 

faces that is non-symmetrical; 

various lighting and shading conditions; and 

a background that contains face-like clutter. 

It has been established with regards to the previous claim that putting text on a distorted 
text ([006]), and that it would have been obvious to combine this with the facial image creation 
of Mortlock. Mortlock discloses the limitations of on-frontal faces (fig 4Fig 3 A, Fig 13), and 
pictures with varied lighting effects (column 13, lines 32-34). It would have additionally been 
obvious to one of ordinary skill in the art to one of ordinary skill in the art of graphic design, to 
manipulate the facial images of Mortlock to make them non-symmetrical. Including these 
features in the HIP would improve its ability to deter non-human users and thus improve it 
offers, and make it more marketable. 



Claim 20 of the application recites: 
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The system of Claim 10 wherein the inputs to generate the image are a 

3D wire model of a generic head and a cylindrical texture map Tm of an arbitrary 

person. 

Tyree discloses a computer means for creating human interactive proof images ([0029], 
[0034], [0072], and [0103]). It does not discuss the manner in which these images are created. 
The creation of 3-d images utilizing wire models was will known in the art of graphics creation 
at the time of invention and the creation of such images is discussed by Mortlock. Mortlock 
discloses a wire frame which is covered in a texture map (fig 3A, column 1, lines 23-25, column 
2, and lines 17-52) used to create an image of a human head/face. This texture map can be 
cylindrical (column 20, lines 11-14) among other shapes. It would have been obvious to one of 
ordinary skill in the art at the time of the invention to combine the HIP system of Tyree with the 
3-D graphics creation technique of Mortlock in order to improve the security and therefore the 
functionality of the system, by using non-textual images which would make it difficult for 
automated scripts which are used to attempting to decode the textual image. 

Claim 23 of the application recites: 

A computer-readable medium having computer-executable instructions for 
creating a test to determine whether a user is a person or a bot, said computer 
executable instructions comprising: 

inputting a 3D wire model of a generic head and a texture map of an arbitrary 
person; and 
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generating a human interactive proof using said generic head model and texture map. 

Claim 23 is essentially the software product for the system of claim 20. It discloses no 
new limitations and as such is rejected for the same reasons as claim 20. 

Claim 24 of the application recites: 

The computer-readable medium of Claim 23 wherein the human 

interactive proof employs an image of a deformed human face in which certain face 

features are at known locations in said image. 

The limitation of known positions of facial features is disclosed by Mortlock as "known 
points" (column 14, table 1, column 13 lines 20-30 & see claim 1 rationale regarding deformed 
images). 

Claim 25 of the application recites: 

The computer-readable medium of Claim 24 wherein a comparison of the 

locations of said features input by a user is made to their actual location in the image and 

is used to determine whether the user is a human or a bot. 

Tyree discloses an intelligence test which offers an image to a potential user and prompts 
the user to input the answer to a question about said image ([0102]), which is then compared 
with the stored answer ([0103]). Mortlock discloses known points in the facial region such as 
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eyes, ears, nose (column 14, table 1). The location of these points in the basic image then would 
be understandable to the program prior to its distortion. After transformations (which are known 
to the program) are applied, the resultant image would not be identifiable by body/facial 
recognition software of the time. It would have been obvious to one of ordinary skill in the art at 
the time of invention to combine the two in order to increase the level of security offered by the 
"test", thus improving the marketability of the software product as a whole. 



15. Claim 14 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over Tyree in view of 
Mortlock and further in view of Mori. 

Claim 14 of the application recites: 

The system of Claim 10 wherein the features of the face comprise the four 
corners of the eyes and the two corners of the mouth. 

Mortlock discusses a method for creating an image of a human head/face from a wire 
model and a texture map (Fig 3 A, column 1, lines 18-21) as well as the known points of eyes, 
ears, chin, etc (column 14, table 1). Regarding the limitation of the facial feature of the four 
corners of the eyes, Mortlock shows texture maps of the two eyes, trimmed at the inner and outer 
corners. In order for such trimming to occur from a larger photograph input, it is inherent that the 
system of Mortlock have the ability to discern the position of the four corners of the subject's 
eyes (Fig 3 A #27). Though the chin and larynx are disclosed as known locations, the two corners 
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of the mouth are not. The Mori, paper discloses an algorithm for locating key points on a human 
body (abstract, page 5 section 5, item 1). It would have been obvious to one of ordinary skill in 
the art at the time of invention to combine the HIP system of Tyree with the facial image of 
Mortlock, and the feature recognition of Mori, in order to create an HIP that offers a wide range 
of potential answer choices, improving its security and thus its functionality. 

Conclusion 

16. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

• "Up to the Challenge: Computer Scientists Crack a Set of Al-based Puzzles" to 
Robinson. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Charles Kyle whose telephone number is 571 272-6746. The 
examiner can normally be reached on 9-5, M-F. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Charles Kyle can be reached on 571 272-6747. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Jalatee Worjloh/ 

Primary Examiner, Art Unit 3685 



